rederive: shadow mode + would-beat-king gate + seed-clobber fix - #95
Open
karpabot wants to merge 1 commit into
Open
rederive: shadow mode + would-beat-king gate + seed-clobber fix#95karpabot wants to merge 1 commit into
karpabot wants to merge 1 commit into
Conversation
- RALPH_REDERIVE=off/shadow/enforce (_rederive_mode); shadow runs + logs the verdict but never blocks a crown (live calibration, no false-reject). - move op_rederive out of judge_submission -> pre-crown gate in router.process_submission AND service.score_and_decide, run ONLY for a would-be king (cost O(king-changes), not O(submissions); closes a DoS). - a failed verdict only WITHHOLDS the promotion (status rederive_withheld); never dethrones the sitting king, never auto-blacklists. Fail-OPEN on every ambiguity (disabled / no canonical data / timeout / too-few points). - drop --seed from the re-run: train.py --seed clobbers BOTH init_seed and data_seed, so any honest init_seed!=data_seed run would false-reject; the full cfg via --config carries both seeds correctly. - env-tunable tolerances (RALPH_REDERIVE_STEP0_TOL/_ABS_TOL/_REL_TOL) so the honest cross-GPU/bf16 spread is calibrated in shadow without a code change. - 26 tests (mode/tols/shadow-wrap/fail-open/seed-guard + router & service gates). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Foundational, zero-live-risk step of the P2→P4→P3 execution hardening. All changes are inert until
RALPH_REDERIVEis set and canonical data is materialized; the container_measurement is untouched (no.py/.yaml/.json/.mdunder a hashed proof dir changes).What
Turns
op_rederive_trajectory(pre-crown proof-of-EXECUTION) from an unusable stub into a calibratable, cost-bounded gate:RALPH_REDERIVE∈ {off (default), shadow, enforce}.shadowruns the re-run + logs the would-be verdict but never blocks a crown, so the honest cross-GPU/bf16 trajectory spread can be collected on the live box with zero false-reject risk before enforcing.judge_submission(ran for every op1-op3-passing bundle = O(submissions) GPU + a DoS surface) into the crown path of bothrouter.process_submissionandservice.score_and_decide, so it runs only for a bundle that would take the crown (O(king-changes)).status: rederive_withheld); it never dethrones the sitting king and never auto-blacklists. Every ambiguity (disabled / no canonical data / timeout / too-few points) stays fail-OPEN.--seedfrom the re-run.train.py --seedclobbers bothinit_seedanddata_seed, so an honest miner withinit_seed != data_seedwould be guaranteed a step-0 false-reject. The full config (both seeds correct) is already passed via--config.RALPH_REDERIVE_STEP0_TOL / _ABS_TOL / _REL_TOLso the band is calibrated from shadow data without a code change.Tests
26 new (
tests/test_rederive_gate.py): mode parsing, tolerance env-threading, shadow-never-gates, fail-open skips, seed-guard, and the router + service crown gates (pass→crown, fail→withhold + king holds, non-decisive→skip). Full suite green apart from 3 pre-existing env-only failures (missingactive_tokens.bin/ GPU-nondeterminism, fail identically onmain).Rollout
Ships off. Next: materialize canonical data on the box (separate, measurement-neutral), then
RALPH_REDERIVE=shadowto calibrate on king09caba0d, then=enforce.🤖 Generated with Claude Code